Remember that for each failure
you will need to find a way to reprocess that record. In our case we are
providing storage and notification but not reprocessing. Because of the
unlimited number of reasons why something could fail, we have no
surefire way to assure a single reprocessing solution.
Adding a Process Failure Method
For this solution to work
we have to look at different ways to implement the processing of the
failed data. The power of Visual Studio 2008 and Visual Basic provides
unlimited possibilities; however, together they provide an implementation model that gives us everything we need with little work.
How we decide to use that
implementation model, however, can enhance or decrease its value.
Internally, .NET supports the notion of the ThreadPool
class. This class provides us with the ability to create managed
threads without having to create the infrastructure code around managing
the threads. This can be both good and bad. If you have a situation
where something needs to run once and only once, this is great. If you
have a situation where you need a large number of threads and you are
not concerned with monitoring those threads or having to deal with
synchronizing or optimizing their wait states by using Sleep, this is
also good. However, if you need to be able to directly monitor a thread
so that if it fails you can restart that thread, using a run once logic
for a given thread would be an issue.
However, even this issue is easy to overcome and should not be seen as a hindrance to the importance and power using the ThreadPool class gives us. We will use the ThreadPool class once again, as we did in our SMTP class.
To implement using the ThreadPool class solution we need two things:
Listing 1 shows the method I have implemented and added to the FileWorker class instance.
Listing 1. ProcessFailures method implementation.
Private Sub ProcessFailures(ByVal args As Object)
Try
If m_ThreadAction.StopThread Then Return
Dim failurefile As StreamWriter = _
My.Computer.FileSystem.OpenTextFileWriter(m_FileWorkerOptions.ProcessError +
"\Failure_File_" _
+ Guid.NewGuid().ToString + ".fri", False)
failurefile.WriteLine(args.ToString())
failurefile.Flush()
failurefile.Close()
failurefile.Dispose()
If (Me.MailEnabled) Then
Dim message As String
Try
message = "Error Processing File Data:[" + args.ToString + "] - "
Catch ex As Exception
message = "Unable to specify data - " + ex.ToString + vbCrLf
End Try
'Send the Email and then move it again to the processed Folder
m_FileWorkerOptions.EmailProperties.Message = message
m_SmtpClient.QueueMail(m_FileWorkerOptions.EmailProperties)
End If
Catch ex As IOException
WriteLogEvent(My.Resources.ThreadErrorMessage + "_" + ex.ToString _
+ "_" + Now.ToString, THREAD_ERROR, EventLogEntryType.Error, _
My.Resources.Source)
Catch ex As Exception
WriteLogEvent(My.Resources.ThreadErrorMessage + "_" + ex.ToString _
+ "_" + Now.ToString, THREAD_ERROR, EventLogEntryType.Error, _
My.Resources.Source)
End Try
End Sub
|
We then use the Visual Basic 2008 My
class to create a text-based streamwriter that we use to write our
record data to before closing the file. Last, we ensure that e-mail
notifications are enabled and, if so, we send an e-mail notification
about the specified data that has failed.
Note that I am creating a new GUID for each file that we produce so that everything is written uniquely.
The Worker Thread
Now we need to update our <ProcessFiles> method to create the worker thread necessary to process the failure request. Listing 2 shows the code.
Listing 2. ProcessFiles code update for new queue thread.
Private Sub ProcessFiles()
Dim LinqSql As LINQSQL = New LINQSQL()
While Not m_ThreadAction.StopThread
If Not m_ThreadAction.Pause Then
Try
For Each TextFile As String In My.Computer.FileSystem.GetFiles( _
m_FileWorkerOptions.Output, _
FileIO.SearchOption.SearchTopLevelOnly, _
m_FileWorkerOptions.FileType)
If m_ThreadAction.Pause Or m_ThreadAction.StopThread Then
Exit For
End If
Try
Dim tmpGuid As String = Guid.NewGuid().ToString()
Dim ProcessFile As String =
m_FileWorkerOptions.ProcessedPath _
+ "\" + tmpGuid + "_" +
My.Computer.FileSystem.GetName(TextFile)
'File is moved so lets read it out of the Output Folder
If (Me.MailEnabled) Then
Dim message As String
Try
message = "Processing File Data:[" + _
My.Computer.FileSystem.ReadAllText(TextFile) + _
"]] From File - "
Catch ex As Exception
message = "Unable to read from file - "
End Try
'Send the Email and then move it again to the
processed Folder
m_FileWorkerOptions.EmailProperties.Message = message
m_SmtpClient.QueueMail(m_FileWorkerOptions.EmailProperties)
End If
Dim records As StreamReader = _
My.Computer.FileSystem.OpenTextFileReader(TextFile)
Dim record As String
record = records.ReadLine()
While (Not record Is Nothing)
Try
Dim insert As Boolean = LinqSql.InsertRecord(record)
If Not insert Then
'Save it
System.Threading.ThreadPool.QueueUserWorkItem( _
New WaitCallback(AddressOf ProcessFailures), record)
End If
Catch ex As Exception
'Save it
System.Threading.ThreadPool.QueueUserWorkItem(
New WaitCallback( _
My.Resources.Source)
End Try
record = records.ReadLine
End While
records.Close()
records.Dispose()
My.Computer.FileSystem.MoveFile(TextFile, ProcessFile, True)
System.Threading.Thread.Sleep(0)
WriteLogEvent(My.Resources.ThreadMessage + _
TextFile, THREAD_INFO, EventLogEntryType.Information, _
My.Resources.Source)
Catch ex As Exception
WriteLogEvent(My.Resources.ThreadErrorMessage + _
"_" + ex.ToString + "_" + Now.ToString, THREAD_ERROR, _
EventLogEntryType.Error, My.Resources.Source)
End Try
Next
Catch fio As IOException
WriteLogEvent(My.Resources.ThreadIOError + "_" + fio.ToString _
+ "_" + Now.ToString, THREAD_ABORT_ERROR, _
EventLogEntryType.Error, My.Resources.Source)
Catch tab As ThreadAbortException
'this must be listed first as Exception is the master catch
'Clean up thread here
WriteLogEvent(My.Resources.ThreadAbortMessage + "_" _
+ tab.ToString + "_" + Now.ToString, THREAD_ABORT_ERROR, _
EventLogEntryType.Error, My.Resources.Source)
Catch ex As Exception
WriteLogEvent(My.Resources.ThreadErrorMessage + "_" + _
ex.ToString + "_" + Now.ToString, THREAD_ERROR, _
EventLogEntryType.Error, My.Resources.Source)
End Try
End If
If Not m_ThreadAction.StopThread Then
Thread.Sleep(THREAD_WAIT)
End If
End While
End Sub
|
Our <LINQSQL.InsertRecord>
method returns a Boolean; however, it also will throw certain errors.
To take the return value and the possible exception thrown into account,
I have added a Boolean validation of the result that then creates a
worker thread in our ThreadPool, passing it our record data.
In our exception handler,
where we will either capture a thrown error or some internal failure, we
again create a worker thread and pass it not only the record
information but also the error message that we received.
Install and Verify
Install and run the
new service. Using the sample files and by saving our data we are at
least assured that we can review that data later on, and if necessary,
copy the data back into our incoming directory and reprocess it when the
server is up again, the network is up again, or whatever issue that
caused the failure has been resolved.
